The background pertaining to the present invention is as follows:
I. Portrait Editing
Because a human portrait is a major component of photographs, as well as one of the image types of the greatest interest, portrait editing is of substantial application merits.
While there exist some portrait editing methods directly working in the two-dimensional (2D) image space such as face beautification (Leyvand, T., Cohen-Or, D., Dror, G., and Lischinski, D. 2008. Data-driven enhancement of facial attractiveness. ACM Trans. Graph. 27, 3, 38:1-38:9.), face swapping without giving source (Bitouk, D., Kumar, N., Dhillon, S., Belhumeur, P. N., and Nayar, S. K. 2008. Face Swapping: Automatically replacing faces in photographs. ACM Trans. Graph. 27, 39:1-39:8.), face enhancement (Joshi, N., Matusik, W., Adelson, E. H., and Kriegman, D. J. 2010. Personal photo enhancement using example images. ACM Trans. Graph. 29, 3, 12:1-12:15.), there is a trend recently in mapping 2D images to three-dimensional (3D) models for more complex applications, such as reshaping of human bodies (Zhou, S., Fu, H., Liu, L., Cohen-Or, D., and Han, X. 2010. Parametric reshaping of human bodies in images. ACM Trans. Graph. 29, 4, 126:1-126:10.), face swapping with source given (Dale, K., Sunkavalli, K., Johnson, M. K., Vlasic, D., Matusik, W., and Pfister, H. 2011. Video face replacement. ACM Trans. Graph. 30, 6, 130:1-130:10.), face component transfer (Yang, F., Wang, J., Shechtman, E., Bourdev, L., and Metaxas, D. 2011. Expression flow for 3D-aware face component transfer. ACM Trans. Graph. 30, 4, 60:1-60:10.), and face animations from large image collections (Shlizerman, I. K., Shechtman, E., Garg, R., and Seitz, S. M. 2011. Exploring photobios. ACM Trans. Graph. 30, 4, 61:1-61:9.), etc.
The specialized 3D model information may enable the portrait editing process to understand fundamentally intricate semantic information contained in an image, and better resolve numerous practical problems caused by ambiguities and occlusions in the destination image.
In view of these researches, the present invention has brought about many new portrait hairstyle editing functions by creating a strand-based 3D hair model for a person's portrait hairstyle.
II. Hair Modeling
Hair modeling is an extensively studied problem in computer graphics, see document below for a comprehensive survey (Ward, K., Bertails, F., Kim, T.-Y., Marschner, S. R., Cani, M.-P., and Lin, M. C. 2007. A survey on hair modeling: styling, simulation, and rendering. IEEE Transactions on Visualization and Computer Graphics 13, 2, 213-234.). The method according to the present invention is most related to image-based hair capture methods thereof, including “Paris, S., Briceno, H., and Sillion, F. 2004. Capture of hair geometry from multiple images. ACM Trans. Graph. 23, 3, 712-719.”, “Wei, Y., Ofek, E., Quan, L., and Shum, H.-Y. 2005. Modeling hair from multiple views. ACM Trans. Graph. 24, 3, 816-820.”, “Paris, S., Chang, W., Kozhushnyan, O. I., Jarosz, W., Matusik, W., Zwicker, M., and Durand, F. 2008. Hair photobooth: geometric and photometric acquisition of real hairstyles. ACM Trans. Graph. 27, 3, 30:1-30:9.”, and “Jakob, W., Moon, J. T., and Marschner, S. 2009. Capturing hair assemblies fiber by fiber. ACM Trans. Graph. 28, 5, 164:1-164:9”. These methods can reconstruct a full 3D model by utilizing multiple hair images taken from different viewpoints, after which the hair model can be rendered in computer graphic applications. Nevertheless, in an application scene of the method according to the present invention, the input is a single image without any knowledge about the hair 3D geometry or the lighting of the scene. Therefore, the method of the present invention aims at generating a plausible high-resolution strand-based hair model for portrait editing applications.
III. Single-View 3D Modeling
Single-view modeling is in general an ill-posed problem. Focusing on specific objects or scenes (Hoiem, D., Efros, A. A., and Hebert, M. 2005. Automatic photo pop-up. ACM Trans. Graph. 24, 3, 577-584.), a method of pop-up (pop-up) for outdoor photos containing ground, sky, and walls is proposed; “Rivers, A., Igarashi, T., and Durand, F. 2010. 2.5D cartoon models. ACM Trans. Graph. 29, 4, 59:1-59:7.” proposes a 2.5D modeling method for cartoon images which conducts smooth interpolation on a small number of cartoon images taken from different views to make it appears like a 3D object; “Oztireli, A. C., Uyumaz, U., Popa, T., Sheffer, A., and Gross, M. 2011. 3D modeling with a symmetric sketch. In Proceedings of SBIM, 23-30.” proposes a method that extracts a 2.5D model of a bilateral symmetric object from a simple user sketch.